Ground-truth effects in learning-based fiber orientation distribution estimation in neonatal brains

Diffusion Magnetic Resonance Imaging (dMRI) is a noninvasive method for depicting brain microstructure in vivo. Fiber orientation distributions (FODs) are mathematical representations extensively used to map white matter fiber configurations. Recently, FOD estimation with deep neural networks has seen growing success, in particular, those of neonates estimated with fewer diffusion measurements. These methods are mostly trained on target FODs reconstructed with multi-shell multi-tissue constrained spherical deconvolution (MSMT-CSD), which might not be the ideal ground truth for developing brains. Here, we investigate this hypothesis by training a state-of-the-art model based on the U-Net architecture on both MSMT-CSD and single-shell three-tissue constrained spherical deconvolution (SS3T-CSD). Our results suggest that SS3T-CSD might be more suited for neonatal brains, given that the ratio between single and multiple fiber-estimated voxels with SS3T-CSD is more realistic compared to MSMT-CSD. Additionally, increasing the number of input gradient directions significantly improves performance with SS3T-CSD over MSMT-CSD. Finally, in an age domain-shift setting, SS3T-CSD maintains robust performance across age groups, indicating its potential for more accurate neonatal brain imaging.


Introduction
Diffusion Magnetic Resonance Imaging (dMRI) is a crucial tool for analyzing in vivo the microstructural organization of white matter fibers in the brain.This technique measures the random motion of water molecules, providing unique insights into brain connectivity and revealing abnormalities undetectable by other modalities both in vivo and non-invasively.The accurate depiction of white matter (WM) fibers using dMRI is particularly important in early brain development stages, where WM is still in maturation.Advanced methods, such as constrained spherical deconvolution (CSD) [30] and multi-shell multi-tissue CSD (MSMT-CSD) [12], are used to reconstruct orientation distribution functions (FODs), enabling both local and global quantitative analyses [4].
Supervised deep learning on high-quality datasets have enabled a growing number of methods [20,21,15,11,16,14,28,26] to predict FODs from the raw diffusion signal [15,11,28] or its spherical harmonics (SH) representation [20,21,14,16], or from a spherical deconvolution model [26].Some of these methods [15,17] have been successfully applied to developing brains for which acquisition times are more constrained, and hence a lower number of measurements is available.In particular, [17] employed a U-Net architecture to predict large patches of FODs from the newborns of the Developing Human Connectome Project (dHCP), using as few as six diffusion measurements and one b0 as input, making it suitable for clinical acquisitions; [19] investigated age-and site-related domain shifts for fiber estimation within rapidly developing populations.The vast majority of these methods either focus on innovating at the architecture level, the learning choices and parameters, or the input format to the network.
The target ground truth (GT) used to be learned from is not often discussed.The majority of the works [20,15,11,16,14,28,26] use MSMT-CSD.However, its use for developing brains has been questioned [23] given the similarity of the response function profiles for each b-value and each tissue at this maturation stage.In fact, the mean signal across b-values of the gray matter (GM) lies within the WM intensities, as opposed to adult data where it lies below, allowing a clear distinction between tissues [23].This distinction between tissues is also age-dependent, making the problem of estimating accurate FODs even more challenging for neonates.Recently, [7] has shown that employing the b1000 shell in the case of newborn (dHCP) data with a method relying on a single non-zero b-value measurement and a b0 7 , named single-shell three-tissue CSD (SS3T-CSD) [5,6], offers superior detection of crossing fibers compared to MSMT-CSD using all the available shells (b400, b1000 and b2600).
In this work, we explored the suitability of SS3T-CSD for deep learningbased FOD estimation in early development stages.We began by evaluating the consistency of GTs generated by MSMT-CSD, commonly used in this context, and SS3T-CSD, which we investigated here.Next, we extensively tested the state-of-the-art deep learning FOD estimation method on 207 newborn subjects, assessing the impact of training with SS3T-CSD compared to MSMT-CSD GT for the first time.Finally, we examined potential domain shift effects due to age by comparing the performance of models trained and tested on cohorts of similar and different ages using both SS3T-CSD and MSMT-CSD GTs.

Datasets and processing
We used neonatal brain MRI data from the 3 rd release of the Developing Human Connectome Project (dHCP) [8], which consists of newborn subjects scanned at the post-menstrual age (PMA) of 26 to 45 weeks using a 3T Philips Achieva scanner.The dMRI data release, having a multi-shell sequence with b-values 0, 400, 1000 and 2600 s/mm 2 and 20, 64, 88 and 128 measurements respectively, had been preprocessed using SHARD [3].The resulting resolution is 1.5 × 1.5 × 1.5 mm 3 , covering a field of view of 100 × 100 × 64 voxels.
Following the approach in [16], we generated a WM mask by integrating the White Matter and Brainstem labels from the provided parcellation of the T2weighted image of each subject, aligning it to the space of the respective dMRI image using ITK-SNAP [31].Voxels with Fractional Anisotropy (FA) values (computed using MRtrix [29]) greater than 0.25 were also included.
For our experiments (detailed in Section 2.3), we selected two subsets, denoted as S 1 and S 2 , totaling 207 unique subjects: -Subset S 42.29, mean: 42.40, SD: 1.07).Each subject's data preparation for the deep learning pipeline (described in Section 2.2) involves sampling single-shell b1000 dMRI images from the full sequence as proposed in [27].These images are normalized by a single b0 image to reduce b-value dependency and then projected onto the corresponding SH spaces to enhance acquisition independence.

FOD estimation learning-based model
The method [17] is based on learning a mapping between the SH representation of the raw diffusion signal and the FOD (Fig. 1

, top).
Ground-truth models.MSMT-CSD extends (single-shell single-tissue) CSD [30] to accommodate multiple shells and multiple tissues; it aims to solve a constrained linear least squares problem using convex quadratic programming [12], in order to find the optimal FOD coefficients for each tissue, requiring acquisitions with at least 3 b-values.On the other hand, SS3T-CSD addresses the optimization problem of finding the optimal FOD coefficients differently: initially, it fixes prior WM coefficients and estimates GM and cerebrospinal fluid (CSF) coefficients; then, it fixes the CSF coefficients and estimates WM and GM coefficients-this two-step iteration is repeated until convergence [5], allowing the method to rely on only two b-value samples (non-b0 and b0).
Network specifications.The network receives a 16 × 16 × 16 patch of SH representations from n sig single-shell diffusion measurements (b1000) as input and outputs the corresponding patch of FODs represented in the same SH basis (SH-L max order 8).We train two models: one targeting FODs estimated from all 300 multi-shell measurements using MSMT-CSD [12], and the other from 88 b1000 and 20 b0 measurements using SS3T-CSD [5], as depicted upper in Fig. 1.
Training.For each experimental setting, training was conducted using the Adam optimizer [18] to minimize the ℓ 2 norm loss of the predicted 45 SH coefficients against the GT FOD SH coefficients, which were generated with MRtrix [29] or one of its forks, MRtrix3Tissue (https://3Tissue.github.io).
During each epoch, 128 patches were randomly extracted from non-empty volumes of each training subject, with each batch containing all 128 patches at a batch size of 1.The training regimen featured an initial learning rate of 1×10 −4 , a dropout rate of 0.1, and used an early stopping strategy.A sliding window technique was employed during inference to process all patches consecutively.Implementation and code availability.Training was performed on an NVIDIA RTX 2080 Ti GPU, each session lasting approximately 24 hours, using PyTorch [22], Lightning [9], and MONAI [2].The code is available at https://github.com/Medical-Image-Analysis-Laboratory/dl_fiber_domain_shift.git.

Experiments
Experiments testing the efficacy of deep learning models in predicting FODs from MSMT-and SS3T-CSD using dMRI data are outlined in Fig. 1.
(ii) Model assessment and ablation study on the number of input directions.Performance between models trained on SS3T-CSD and those trained on MSMT-CSD was compared.This comparison was also performed across a different number of input directions to the network.The network input has n sig measurements (n sig ∈ {6, 15, 28, 45}) from the b1000 shell, each projected onto an SH basis with L max ∈ {2, 4, 6, 8}.In this phase, the data subset S 1 was divided into groups of 70 for training, 15 for validation, and 15 for testing.
(iii) Age-related domain shifts.To explore age-related variations within each dataset, we conducted age-specific training and testing within and across these ages [19].As mentioned in Section 2.1, in the dHCP dataset, subjects in S2 were selected and divided into two age groups, denoted as early and late, respectively.Each age group consists of 65 subjects, further partitioned into splits of 40 for training, 10 for validation, and 15 for testing.We fixed the number of input measurements to 15 (corresponding to SH order 4) and trained separate models on the early and late age groups.Each model was then tested on its own age group (self-testing) as well as on the other age group (cross-age testing).

Evaluation metrics
Quantitative validation was performed based on the agreement rate (AR) [17] between the number of fibers estimated by the number of peaks extracted using Dipy [10] (with a mean separation angle of 45°, a maximum number of 3 peaks and relative peak threshold of 0.5), the angular error (AE) between those peaks [17], and the apparent fiber density (AFD) [24] error.For the AFD error, we report the mean absolute percentage error (MAPE) as the magnitude of the SS3T-CSD-derived FODs is higher than that of MSMT-CSD.

Results
Firstly, we show a qualitative comparison between the FOD reconstructed using the two GT methods for a neonate with 40 weeks PMA (Fig. 2), where we clearly see missing crossings in MSMT-CSD, likely due to the overestimation of the gray matter compartment, compared to SS3T-CSD.

Ground-truth consistency
Table 1 provides a comparison between the two GT methods.MSMT-CSD shows a higher agreement rate for single fibers but similar angular errors when compared to SS3T-CSD.In contrast, in the 2-fiber configuration, SS3T-CSD outperforms MSMT-CSD in agreement rate and exhibits slightly better angular errors.
It is fair to mention that SS3T-CSD employs the minimum number of directions given the SH-order of the FOD (44 directions for 45 coefficients), compared to 140 directions by MSMT-CSD.Hence, the SS3T-CSD consistency errors reported here represent the upper bounds of expected errors.This is particularly relevant for 3-fiber configurations, which are more susceptible to noise and thus less reliable.Importantly, closer examination of the confusion matrices (Table 2) reveals that the proportion of multiple fibers estimated by MSMT-CSD is around 23%, significantly lower than the literature-reported values of over 60% [13,25], whereas SS3T-CSD estimates this proportion to be 61%.

Model assessment and number of input directions effect
Fig. 3 illustrates the comparative results in terms of agreement rate, angular error, and AFD error across varying numbers of input gradient directions, using MSMT-CSD and SS3T-CSD GTs, respectively.We observe that SS3T-CSD consistently shows higher agreement rates and lower angular errors for multiple fibers compared to MSMT-CSD across all input directions, except for 6 directions where they are comparable.Increasing the number of input directions improves the network's accuracy in predicting multiple fibers more significantly with SS3T-CSD than MSMT-CSD.Specifically, SS3T-CSD achieves an agreement rate of around 70%-76% for single and 2-fiber populations with 28-45 input directions, while MSMT-CSD's 2-fiber population agreement rate remains around 30% compared to 88% for single fibers, even with 45 input directions.However, for single fiber populations, MSMT-CSD produces more accurate predictions.For AFD, SS3T-CSD shows lower errors across all input levels compared to MSMT-CSD, with the former exhibiting lower error variance.This can be attributed to the fact that AFD is computed using the same b-value as the input b-value measurement to the network for SS3T-CSD, whereas it is computed with all three shells in MSMT-CSD.

Age domain shift
Fig. 4 presents the effects of age-specific training and testing on model performance using MSMT-and SS3T-CSD GTs.We first notice that when testing in the early-stage cohort, SS3T-CSD cross-age training is more robust to domain shift across all metrics, including single-fiber populations AR and AE, compared to MSMT-CSD.In fact, the drop for instance in 2-fibers agreement rate goes from around 28% to 18% for the latter, but only from 55% to 50% for the former.However, for the configuration when tested in the late-stage cohort, no significant difference can be observed between the two GTs, except for AFD where SS3T-CSD seems more robust to domain shift.In general, an increased variance is observed in cross-testing scenarios in for both SS3T-CSD-and MSMT-CSDtrained models, especially for AFD.

Conclusion
We have demonstrated the differences in the training of deep learning models using MSMT-CSD and SS3T-CSD for FOD estimation in neonatal brains.Compared to MSMT-CSD, SS3T-CSD improved accuracy and reliability in multiple fiber configurations, which is a major bottleneck in diffusion MRI and, in particular, for rapidly developing brains [1].However, there is a drop in performance in single-fiber populations compared to MSMT-CSD, potentially due to higher values in gray matter compartments in GT MSMT-CSD that smoothens the overall estimated FODs, resulting in less predicted multi-fibers voxels, and viceversa for SS3T-CSD and multiple fibers predictions.More b1000 measurements for SS3T-CSD can potentially lower this effect.Furthermore, SS3T-CSD showed robust performance across different age groups, reducing the performance drop caused by age domain shifts.Future work will focus on evaluating SS3T-CSD in other pediatric populations, such as fetuses and babies, and exploring downstream performance in tractography.We will also investigate a variety of models beyond U-Net and different strategies to address domain shift challenges through data harmonization and domain adaptation techniques.
Ethics Statement.This study uses pre-approved dHCP data by UK Health Research Authority (REC reference: 14/LO/1169), requiring no additional ethical approval.

Fig. 1 .
Fig. 1.Workflow overview.b1000 dMRI images are sampled from the full series, normalized by b0 and projected into SH space.These images are inputs to a deep learning model predicting GT FODs, using either MSMT-CSD (300 meas.) or SS3T-CSD (108 meas.).The experiments assess: (i) consistency of GT algorithms, (ii) impact of input quantity on model performance, and (iii) model effectiveness across different neonatal developmental stages.Illustrative brain images are from dHCP.

Fig. 2 .
Fig. 2. Qualitative comparison of coronal slices of the FODs used as GT for the model, reconstructed with MSMT-CSD and SS3T-CSD, respectively, from the dMRI scan of a subject at 40 weeks PMA.Both gray matter and white matter compartments are displayed on the fractional anisotropy (FA) image computed from dMRI data.FOD estimation and visualization were performed with MRtrix [29].

Fig. 3 .
Fig. 3. Comparison of performance of the models based on incrementing numbers of input directions, using GT: (a) MSMT-CSD; (b) SS3T-CSD.AR and AE under different fiber number configurations and AFD Error are depicted.

Table 1 .
Quantitative comparison of the consistency of GT FODs generated with MSMT-CSD and SS3T-CSD.Agreement Rate (AR, %) and Angular Error (AE, • ) for each fiber number configuration, and AFD MAPE (∆AFD, %) are listed, together with the number of measurements (nm) used and the corresponding b-values.AE is computed among fibers with agreed peak predictions only.

Table 2 .
Confusion matrices in the number of peaks agreement, normalized over all populations (in %) for MSMT-CSD and SS3T-CSD ground truth consistency.
Comparison of performance metrics for FOD estimation models trained and tested on early and late developmental stages using MSMT-CSD and SS3T-CSD ground truths.The agreement rate, angular error, and AFD error are depicted across models trained on early (M early ) and late (M late ) age groups.